UNCLASSIFIED 



AD NUMBER 


AD411983 


NEW LIMITATION CHANGE 
TO 

Approved for public release, distribution 
unlimited 


FROM 

Distribution authorized to U.S. Gov't, 
agencies only; Administrative/Operational 
Use; Jun 1963. Other requests shall be 
referred to Hq Electronic Systems 
Division, Attn: ESAT, L.G. Hanscom Field, 
Bedford, MA. 


AUTHORITY 


CFSTI per ESD ltr, 10 Aug 1965 


THIS PAGE IS UNCLASSIFIED 














UNCLASSIFIED 

u) 411983 


DEFENSE DOCUMENTATION CENTER 

FOR 

SCIENTIFIC AND TECHNICAL INFORMATION 

CAMERON "TATION. ALEXANDRIA. VIRGINIA 



UNCLASSIFIED 
















NOTICE: When government or other drawings, speci¬ 
fications or other data are used for any purpose 
other than in connection with a definitely related 
government procurement operation, the U. S. 
Government thereby incurs no responsibility, nor any 
obligation whatsoever; and the fact that the Govern¬ 
ment may have formulated, furnished, or in any way 
supplied the said drawings, specifications, or other 
data is not to be regarded by implication or other¬ 
wise as in any manner licensing the holder or any 
other person or corporation, or conveying any rights 
or permission to manufacture, use or sell any 
patented invention that may Id any way be related 
thereto. 



ESD-TDR-63-403 


j 


GO 

O 

iH 

rH 


CJ> 

Q 

C~Jt 


>~ 

CQ 


O 


crt 

UJ 

CD 

c "> 




-JZ 

CjO 


PSVCHOACOUSTIC SPEECH TESTSs A MODIFIED RHYME TEST 

TECHNICAL DOCUMENTARY REPORT NO. ESD-TDR-63-403 
JUNE 1963 


Arthur S. House 
Carl Williams 
Michael H. L. Hecker 
Karl D. Ki >er 


DECISION SCIENCES ABORATORY 
ELECTRONIC SYSTEMS DIVISION 
AIR FORCE SYSTEMS COMMAND 
UNITED STATES AIR FORCE 
L. G. Hartscom Field, Bedford, Massachusetts 




Project 7684 




(Prepared under Contract No, AF 19 (628)—382 by Bolt Beranek and Newman Inc. 
Cambridge, Massachusetts.) 





When US Government drawings, specifications or other data are used for any purpose other 
than a definitely related government procurement operation, the government thereby incurs 
no responsibility nor any obligation whatsoever; and the fact that the government may have 
formulated, furnished, or in any way supplied the said drawings, specifications, or other 
dota is not to be regarded by implication or otherwise, as in any manner licensing the 
holder or any other person or corporation, or conveying any reighb or permission to manu¬ 
facture, use, or sell any patented Invention that may in any way be related thereto. 


This document may be reproduced to satisfy official needs of US Government agencies. No 
other reproduction is authorized except with permission of Hq Electronic Systems Division, 
Attn: ESAT, L. G. Hanscom Field, Bedford, Massachusetts. 

Do not return this copy. Retain or destroy. 


Copies available at Office of Technical Services, Department of Commerce. 

Qualified requesters mey obtain copies from Defense Documentation Center (DDC). 
Orders will be expedited if placed through the Librarian or other person designated 
to request documents from DDC. 


PSYCHOACOUSTIC SPEECH TESTS: 
A MODIFIED RHYME TEST 


ABSTRACT 

A multiple choice, easily scored speech test has been developed 
and evaluated. It was found that the speech Intelligibility scores 
obtained with this test remain consistent for a given communication 
system when tested nearly daily for a period of one month using 
enlisted personnel as test listeners. 



ill 


SECTION 1 
INTRODUCTION 

There is a pressing need in some branches of the military services 
to routinely evaluate the ability of voice communication systems to 
transmit intelligible speech. Ideally this "evaluation" should be 
done in a manner that provides a direct quantitative measure of the 
ability of operational personnel to understand spoken messages over 
any system. The testing procedures should require but a few minutes, 
little or no equipment, and untrained listeners. This method or 
technique should, of course, be reliable and should supplement the 
less direct pure-tone "calibration" and other electronic test pro¬ 
cedures typically used to check the electronic functioning of a 
speech system. 

The most obvious methods to use are the so-called speech "articulation" 
or speech "intelligibility" tests. Such tests have been used for many 
years in the laboratory for the evaluation of the performance of 
speech communication systems. 

Unfortunately, the literature dealing with speech intelligibility and 
articulation testing gives ample evidence of the tedious and time- 
consuming nature of these methods. Testing methods that are convenient 
to administer and score, and, at the same time, are short and re¬ 
liable do not appear to be available. The present investigation 
represents an attempt to develop and evaluate an instrument that can 
be used routinely by relatively naive talkers and listeners, that is, 
operational personnel, to evaluate the condition of communication 
systems. 

The format used is similar in general respects to that described by 
Fairbanks as a rhyme test. 1 It differs somewhat in its composition. 


I. G. Fairbanks, Test of phonemic differentiation: The rhyme test. 

J. Acoust. Soc. Am. 30, 596-600 (1958). 
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consisting of six alternate word lists in which no account is taken 
of word familiarity, nor of the relative frequency of occurrence of 
sounds in the language, nor of the orthographic constraints imposed 
on the Fairbanks' materials. In addition, the present materials 
use variable phonemic elements in word-final as well as word-initial 
position. 

The major innovation in the present investigation has to do with the 
task required of the listener. The listener has available to him 
a closed set of six alternatives from which he is required to select 
his identification of the message. The response sets are generated 
automatically by the structure of the six forms of the articulation 
test. In actuality the listener has available a complete description 
of the overall message set and he is asked to select from it the 
word being transmitted. This technique has the advantage of elim¬ 
inating the learning time required in the usual speech intelligibility 
tests where the listeners must be trained for a number of days before 
they become thoroughly familiar with all the words used in the speech 
tests. Since little or no learning is required it should be possible 
to make use of relatively naive listeners. In addition, it is be¬ 
lieved that the test, with suitable "scramblings" or re-arrangements 
of the test items, can be administered over and over again to the 
same listeners without the scores changing as a result of the listen¬ 
ers finding the test "easier" with practice. This, of course, is an 
important problem for the repeated, periodic testing of speech systems 
in the field. 

The materials to be described retain a high degree of phonemic balance 
from test form to test form, and, therefore, lend themselves to analytic 
appraisals of transmission in terms of speech sound elements. The 
identification of sound confusions in a speech link constitutes 
valuable diagnostic information that can lead to the identification 
of specific malfunction. 
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SECTION 2 


THE TESTS AND ANSWERS SHEETS 

Six lists of American English words were used as test materials, 
each list consisting of 50 monosyllabic words. As a rule the words 
are of the form consonant-vowel-consonant (CVC); some few words take 
the form CV or VC. The lists were constructed in such a way as to 
form 50 ensembles, each composed of six related words. Any given 
ensemble is characterized by one vowel that is the nucleus of each 
word; all of the words in a given ensemble either are initiated or 
terminated by the same consonantal phoneme or phoneme cluster. 

An example of an ensemble in which each word contains the vowel /i/ 
and ends with the consonant /t/ is 

meat, feat, heat, seat, beat, neat. 

Each word in this ensemble appears in a different te3t list, while 
the ensemble Itself is presented to the listener aa a finite, response 
set. 

The ensembles used in this study include 25 in which the final con¬ 
sonant is constant and 25 in which the initial consonant is constant. 
Elemeu 1 that were varied in initial position in words number 20 
(including the absence of a consonant, Indicated by #), but six of 
these did not appear in all test forms. Similarly, 20 consonantal 
elements were used as variables in final position, but only 10 appear 
in all test forms. In all, 23 variable elements are used in the test 
material, 13 occurring in initial position in every test form, 10 
occurring finally in every form, and nine occurring both initially 
and finally in every test form. The number of times variable elements 
occur is given in Table 1. 
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Table 1. Frequency of occurrence of variable consonantal elements. 

The symbol # indicates the absence of a consonant. Items 
marked * occur in all six test forms in word initial 
position; items marked + appear in all test forms in word 
final position. 


t*+ 

29 

r* 

12 

k*+ 

27 

w* 

9 

s*+ 

26 

V 

6 

n+ 

24 

0 

5 

P*+ 

23 

# 

5 

b*+ 

20 

e 

5 

d*+ 

19 

z 

4 

i*+ 

17 

/ 

3 

m*+ 

16 

tf 

3 

f* 

16 


3 

S*+ 

h* 

14 

12 

3 

2 
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The nonvarying consonantal elements used in the word lists are 
shown in Table 2j Table 3 shows the distribution of vowel sounds 
according to ensemble structure. 

Table 2. Frequency of occurrence of constant nonvocalic elements 
in ensembles, according to position in word. 



Initial 

Final 


Initial 

Final 

p 

5 

2 

f 

1 

0 

b 

3 

0 

s 

5 

0 

t 

2 

3 

Z 

1 

4 

d 

2 

2 

r 

1 

0 

k 

3 

2 

h 

1 

0 

g 

0 

1 

st 

0 

2 

m 

1 

1 

nt 

0 

1 

n 

0 

3 

rk 

0 

1 

9 

0 

1 

# 

0 

2 


Table 3. Frequency of occurrence of vowel sounds, according to 
consonantal structure of ensemble. 

Variable Vowel 

iI£esaoAOOIU 

Initial 

Consonant 26431312111 


Final 

Consonant 55055005000 
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The spoken lists . Since the individual words were to be identified 
by the listeners as a particular item in a known response set, that 
is, as one of six words, the order of the items in each of the six 
test lists is best considered as an order of ensembles. The 50 en¬ 
sembles, therefore, were arranged arbitrarily in three orders, pro¬ 
viding three forms of each of the six test lists (see Appendix I). 
These 18 word lists were read by two adult male talkers with exper¬ 
ience in the recording of materials for use in listening tasks. 
Between each word approximately 5 sec was provided for the listener's 
response. The words were spoken without instrumental monitoring, 
the talkers attempting to maintain a constant level of vocal effort 
throughout each list. 

The lists were recorded on magnetic tape on a high-quality system 
arranged in a two-room sound-treated recording facility. The major 
components of the system were an Altec 21-D microphone system and 
an Ampex 351-2 tape recorder operating at 15 lps. The talkerB were 
seated comfortably and the microphone was positioned 12 in from their 
lips. The recorder gain was adjusted so that talker levels were 
approximately equal. 

The response forms . The listener always was provided with a response 
form that contained the 50 word ensembles in the proper order of the 
presentation. Each ensemble was enclosed in a rectangular box with 
its six words on two lines, as for example. 


meat 

feat 

heat 

seat 

beat 

neat 


Listeners were instructed to draw a line through the item heard. A 
copy of the written instructions presented to the listeners is in¬ 
cluded in Appendix H. 


- 6 - 



In addition to varying the order of ensembles on the response form 
to correspond to the order of word presentation in the three lists 
recorded by the talkers, the arrangement of items in each response 
ensemble was permuted systematically so that each word appeared once 
in every location. (This concern with possible spatial biases was 
motivated by considerations relevant to eventual mechanization of 
tests of this type, since button-pressing activities may be biased 
by spatial location.) In other words, six different response sheets 
were available for each form of the test; only three orders of en¬ 
sembles occurred on the response sheets, but similar ensemble orders 
differed in the arrangement of the words within ensembles. Samples 
of the six response sheets are included in Appendix II. 
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SECTION 3 


EXPERIMENTAL EVALUATION OP TEST 

An experiment was performed to determine the general reliability 
and acceptability of this new test when administered under a 
wide variety of voice conditions to Air Porce enlisted personnel. 

In this experiment we wished to determine whether: (a) the 
different forms of the tests differed in terms of difficulty 
under different testing conditions; (b) the listeners required 
any special training in order to use the tests; (c) the scores 
of a group of listeners changed significantly with repeated 
exposure to the same test forms; and (d) there were nonfunctional 
or "dead wood" items present In the tests. 

The word lists were presented to listeners at six ratios of 
signal and noise. The master recordings of the lists were played 
on a high-quality tape system and mixed appropriately with speech¬ 
shaped noise before presentation to the listeners under an earphone. 
The arrangement of the stimulus presentation equipment is shown 
in block diagram form in Pig. 1. All listening was done monaurally 
with TLH-39 (10-ohm impedance) earphones; a dummy earphone was 
on the opposite ear. Subjects were seated comfortably In arm- 
desk chairs located in a relatively quiet classroom at an Air Porce 
Base. 

The speech output of an Ampex 601 playback was constant at a level 
of approximately 80 db SPL, and the noise level was manipulated 
to provide signal-to-noise ratios of +4, 0, -4, -8, -12, and -16 
db. These levels were arrived at during preliminary training 
sessions. 

The listeners were 18 enlisted men on active duty in the U. S. Air 
Force. The ages of the listeners ranged from 18 to 28 years with 
a mean of 23-8 years. The hearing of all listeners was tested with 
a standard clinical audiometer. Two subjects had slight deviations 
of acuity from normal threshold for pure tones at 4 and 6 kcps, 
but not sufficient for rejection as subjects; no other abnormalities 
were noted. 
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FIG. I BLOCK DIAGRAM OF EQUIPMENT FOR STIMULUS PRESENTATION 













In order to cancel out various biasing effects on the listeners' 
responses, the order of presentation of the test materials was 
regulated by a schedule based on a quasi-symmetrical, 12 x 12 
latin square. In this schedule each row of the matrix contained 
each of the six test lists produced by each of the two talkers,* 
such a sequence of lists, that is, one row of the matrix, was 
equivalent to two listening sessions. Within each row, however, 
the signal-to-noise ratio at which lists were presented was varied, 
as was the (ensemble) order within lists and the arrangement of 
response ensembles, that is, response 3heets, in the hands of the 
listeners. The arrangement was not completely counterbalanced, 
and the listeners did not have an opportunity to hear both talkers 1 
versions of every list at each signal-to-noise level. All lists 
were presented at every level, however, and each talker was heard 
at every level. 

In practice, the 18 listeners were divided into four groups, two 
groups consisting of four men and two consisting of five men. The 
listening schedule of each group of subjects was started on a 
different row of the matrix, and progressed thereafter from row 
to row until all of the conditions of the entire schedule had been 
experienced. The testing procedures were accomplished over a 
period of approximately 30 days. The listeners did not score their 
own response sheets after each test was administered, and, hence, 
were not aware of their successes and failures as the experiment 
progressed. 

Results 

The general results of the testing procedures are displayed in 
Pig. 2. Responses are averaged over 18 listeners and the percent 
correct is plotted versus the six presentation levels. The 
average responses to the various test forms, A-P, are identified 
for each of the two talkers versions of these materials. Solid 
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lines connect the level of average response to each talkers' 
materials at the various levels. 

This presentation indicates clearly that, on the average, the 
listeners found the tests recorded by one talker more intelligible 
than those recorded by the other talker. This disparity iB 
consistent from level to level; although at high levels of correct 
response the difference between the two sets of materials is 
reduced, the recording of the two talkers elicit responses at 
different levels of success even in the most favorable signal-to- 
noise condition. 

The progression from poor response at unfavorable S/N levels to 
increasingly better levels of success as the S/N conditions are 
improved is approximately as expected. The response improvement 
over the linear portion of the curves, however, is about 5# for 
every 1 db of improvement in S/N level. This increase in 
successful response is at a faster rate than reported by Fairbanks 
with comparable materials in a different test format. The six 
nominal S/N levels, selected on the basis of pilot data runs, can 
also be seen to be inefficient; perhaps smaller separations 
between levels and a general shift toward larger negative ratios 
would have been more suitable. 

The average responses to the six test forms at various levels are 
tabulated in Table 4, part A. The average levels of response 
to the six forms of the test, the column means, are seen to be 
highly similar, as suggested by the lack of correlation between 
various tests from level to level in Fig. 2. A statistical test 
of significance, of course, cannot reject the hypothesis of no 
difference between test form means. 
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PERCENT CORRECT RESPONSE 



FIG. 2 CORRECT RESPONSES AVERAGED FOR 18 LISTENERS 





Table 4. Average percent correct response for 18 listeners arranged 
according to test form and signal-to-noise level of presen¬ 
tation. Part A: actual data. Part B: average 'corrected' 
to account for talker differences. 

A 


S/N 

Levels Test Forms 


(dB) 

A 

B 

C 

D 

E 


Mn 

SD 

-16 

32 

43 

33 

34 

33 

38 

35 

4.0 

-12 

53 

51 

50 

57 

55 

60 

54 

3.4 

-8 

74 

81 

69 

83 

66 

79 

75 

6.1 

-4 

90 

86 

90 

84 

93 

85 

88 

3.1 

0 

96 

95 

94 

90 

93 

92 

93 

2.2 

+4 

98 

98 

97 

92 

98 

95 

96 

2.2 

Mn 

74 

76 

72 

73 

73 

75 

74 



B 


-16 

31 

38 

36 

32 

34 

40 

35 

3.5 

-12 

54 

56 

52 

54 

56 

58 

55 

2.1 

-8 

78 

75 

76 

76 

75 

77 

76 

1.2 

-4 

90 

86 

90 

88 

88 

86 

88 

1.8 

0 

96 

96 

94 

93 

95 

93 

94 

1.4 

+4 

98 

98 

97 

96 

97 

97 

97 

0.8 
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The row (that is, level) means in this table are essentially as 
forecast by Pig. 2, but the magnitude of the standard deviations 
may be surprising. These SB's reflect the variation seen in 
the form scores at a given level; this variation is attributable 
to the incomplete balance in the test administration. The entry 
81# for Form B at level -8, for example, is based only on responses 
to talker CW, and, therefore, is an inflated estimate. The lower 
portion of the table, part B, is an estimation of the scores that 
would have resulted if complete balancing had been achieved. 

Missing entries have been estimated by averaging other forms. This 
reconstruction emphasizes the stability of responses from level 
to level, and would result in smaller values of SD. 

A test of the row means in Table 4, part A, indicates, as expected 
that statistically significant differences do exist between these 
values. For these data, a difference of about 10# is required at 
the 5# level of confidence. In Pig. 3 the form-to-form variation 
suggested by Table 4, part B, is displayed graphically for the 
four most unfavorable S/N ratios. The average correct response 
for the 18 subjects over six S/N levels is Included. 

Table 5- Average percent correct responses for 18 listeners 

arranged according to listening groups. Data arranged 
over levels, test forms, and talkers. 


Group 


Individuals 


Mn 

1 

71 

74 

72 

73 

76 

73 

2 

73 

72 

73 

75 

- 

73 

3 

77 

72 

74 

76 

75 

75 

4 

73 

73 

76 

75 

- 

74 
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FIG. 3 


AVERAGED RESPONSES SHOWING 
TEST FORM CHARACTERISTICS 






The behavior of the listeners follows patterns commonly found in 
behavioral experiments. The average percent correct response 
for each of the 18 listeners wsb close to the grand average of 
74$. The standard deviation of individual averages was 1.7; the 
range was 6, varying from 71$ to 77$. The 18 listeners whose 
data are analyzed were rotated through the listening schedule in 
four groups. The group averages and individual averages are 
reported in Table 5. 

A rapid indication of the differences, albeit small, that exist 
amongst the listeners is afforded by examining the mean performance 
level of each subject on each row of the schedule matrix. Each 
matrix row included two talkers versions of the six test forms, 

p 

and included tests at all six S/N ratios. The Friedman 
nonparametric test of significance was applied to the ranks of 
the matrix means for each listener and indicated that the 
differences among the listeners' averages would occur by chance 
with a probability less than 0.01. The test indicates essentially 
that substantially high correlation exists between the performance 
of the individuals in the listening group from row to row in the 
listening schedule. Rank-difference correlation coefficients 
for the 18 listeners calculated between individual rows of the 
schedule matrix and the grand average for each listener ranged 
for 0.40 to 0.85 (with 7 out of the 12 coefficients exceeding 
0.75). 

These findings suggest that a high degree of temporal stability 
exists in the materials. Figure 4 shows the average response 
of the 18 listeners at the various S/N levels on their 'nominal' 
first and last testing days. Repeated exposure to the materials 
did not change the average response levels in any appreciable way. 


2. S. Siegel, Nonparametric Statistics for the Behavioral Scientist 
(New York: MeGraw Hill, 1956). 
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Some additional insight into the variability of the responses 
of individual listeners was sought by examining the scores of 
individual tests. For example, the 18 listeners heard talker 
AH in 12 tests administered at a S/N ratio of -8db. (The forms 
included in these 12 tests were A, C, S and F.) The standard 
deviation of the 12 scores provided by each listener was computed 
and these were found to range from 5.6 (4 listeners!) to 10.7$; 
the average of the 12 SD's was 7.8. 

Table 6. Average percent correct responses arranged according 
to test forms and position of phonetic variable. 



A 

G 

C 

D 

E 

F 

Initial 

78 

81 

79 

79 

78 

81 

Final 

68 

71 

66 

67 

68 

67 
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AVERAGE RESPONSES SHOWING 
TEMPORAL STABILITY 





Teats of Initial and Final Consonanta 


Since each of the six alternate forms consists of 25 items in 
which word-initial consonant sounds are under test and 25 items 
in which word-final sounds are under teat, it may be of interest 
to examine these two halves separately. In general the distribution 
of sounds under test is similar in these two halves, the major 
exceptions being consonants that occur in English only in one or 
the other position. The sound h, for example, does not occur in 
word-final position in the language; similarly for the sound w. 

A few sounds are used in only one half of the test to provide a 
needed ensemble, z_ in final position, for example. 

The average percent of correct response for the two halves of the 
various forms of the test is given in Table 6. The tabulated data 
indicate that the halves of each form behave very similarly, the 
variation in average correct response from form to form being very 
low. The data demonstrate the generally known fact that Initial 
consonants are more recognisable than final consonants, the average 
correct response to the latter being 68# and to the former, 80 #. 

The question of whether either half of the test would be sufficient 
for testing purposes seems worthy of investigation. 

The responses of the listeners can be analyzed to reveal the degree 
of success with which various consonantal elements were identified. 

In Table 7 are tabulated the average percent of correct responses 
associated with various consonantal elements, as Indicated. The 
results are shown separately for consonants in Initial and final posi¬ 
tions. These data indicate that while the superiority of initial 
consonants over final consonants is generally maintained, there are 
instances where no appreciable difference is found. They also show 
a tendency for the voiceless form of cognate sounds to be heard more 
correctly than the voiced form, as for example, £ > b, £_ > v, l > d. 
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This last finding is of particular interest since it is at variance 
with classical descriptions of speech sound discrimination results. 

The data reported by Fletcher^, for example, indicate that, on the 
average, the voiced forms are perceived more readily than unvoiced 
forms. The Fairbanks' (l) rhyme test data support this general ob¬ 
servation (but there are some inversions in his Table VI, notably 
s > z and f > v). 

The data on Table 7 are also at variance with the descending order of 
percent correct identification of sounds reported by Fairbanks (l). 

He found, for example, that when the general level of identification 
for his materials as a whole was 50tf, the nasal consonants m and n 
were correctly identified at the highest rate while a, £ and t were 
poorly identified. Examination of Table 7 reveals that these sounds 
in initial position (Fairbanks' materials were all in this position) 
rank inversely to the Fairbanks data. The data in Table 7, of course, 
are averaged over six signal-to-noise ratios but examination of the 
data in detail reveals that the general order Is maintained from level 
to level. When the 14 sounds in Fairbanks' Table VI that appear in 
Table 7 are compared a rank-difference correlation coefficient of 
-0.53 results, indicating a general Inversion in order. It is possible 
that disagreements between our results and those reported by Fairbanks 
and others may be at least partially due to differences in the char¬ 
acteristics of the masking noises used in the various experiments. 

In contrast to the earlier studies which used noise of equal intensity 
at all frequencies, the present study used a masking noise that was 
shaped to approximate the long-time speech spectrum envelope. 

Confusion Matrices 

A detailed analysis of the confusions manifested by the listeners' 
responses is not undertaken here. In general, it can be said that 
these confusions follow theoretical expectations and previous results 
to a marked degree. 


3. H. Fletcher, Speech and Hearing in Communication (New York: 
D. Van Nostrand Co. Inc., 1953)• 
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Table 7. 


Average percent correct responses arranged according to 
phonetic elements. Initial (I) and final (f) occurrences 
of phonetic elements are tabulated separately. The symbol * 
indicates that a sound in a given position is not in all 
six forms of the test; the symbol *# Indicates that the 
sound occurs in but one form of the test in the position 
indicated. 



I 

F 


I 

F 

p 

82 

56 

t/ 

- 

83 

t 

91 

79 




k 

83 

65 

* 

85 ** 

69 

b 

72 

57 

m 

67 

38 

d 

75 

64 

n 

66** 

64 

S 

77 

70* 

9 

- 

57 

f 

86 

74* 

w 

79 

— 

e 

81** 

56* 

r 

69 

68 

s 

98 

96 

l 

71 

74 

/ 

61* 

- 




h 

70 

- 

# 

* 

70 

79 


- ** 

, * 




V 

61 

65 




§ 

66** 

1 1 ** 

44 




z 

- 

91 
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Confusion matrices for the 50 ensembles averaged over talkers, 
listeners, test forms and six signal-to-noise ratios make up the 
body of Appendix III, These matrices reproduce the number of 
responses made to each word in every ensemble. The arrangement 
of words is arbitrary and the rows do not correspond to test forms. 
Since there was an imbalance in the original schedule for adminis¬ 
tering the tests the row totals are not the same, but vary between 
432 and 437; the total number of responses in each matrix is 2604. 

The average correct response to each word has been derived from 
the matrices in Appendix III and is reported a3 a percentage in 
Appendix I. 
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Appendix I 


The test materials are presented as a list of ensembles, arranged 
in two parts. Ensembles are read across the rows. The first 25 
ensembles are formed by variation of final (phonetic) elements, and 
are tabulated alphabetically by initial letter. The second 25 en¬ 
sembles are formed by variation of initial elements and are ordered 
alphabetically by final letter. 

The columns headed by letters A-F represent forms of the test. The 
right hand columns headed by the numbers 1-3 indicate the ensemble 
(and test) orders used in the three versions (scrambles) of each test 
form as used in the present experiment. 

The number in parentheses that follows each word is the percent of 
correct responses averaged over l8 subjects, 6 levels and 2 talkers. 
The numbers in parentheses following form headings are averages of 
correct responses over 25 wordsj the numbers in parentheses following 
each ensemble number are averages of correct responses for each 
ensemble. 
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Section 1: Variations of Pinal Element 


Forms 



A( 68 ) 

B(71) 

C ( 66 ) 

D(67) 

E( 68 ) 

F(67) 

1(77) 

bat( 70 ) 

bad( 68 ) 

back(55) 

bass( 98 ) 

ban( 70 ) 

bath(60) 

2 ( 56 ) 

bean(47) 

beach(83) beat(76) 

beam(l4) 

bead(55) 

beak(59) 

3(73) 

bun(72) 

bus(99) 

but(63) 

buff(71) 

buck( 56 ) 

bug(75) 

4(64) 

came( 38 ) 

cape(55) 

cane( 50 ) 

cake(72) 

cave( 69 ) 

case(97) 

5(74) 

cut(8o) 

cub( 63 ) 

cuff(70) 

cup(66) 

cud(70) 

cuss( 98 ) 

6(70) 

dig(7l) 

dip(80) 

did(62) 

dim(65) 

dill(67) 

din(74) 

7(61) 

duck(76) 

dud(68) 

dung(63) 

dub(26) 

dug(64) 

dun(68) 

8(76) 

fill(73) 

fig(81) 

fin (71) 

fizz(94) 

fib(54) 

fit( 82 ) 

9(63) 

hear(73) 

heath(54) heal(62) 

heave(71) 

heat(86) 

heap(30) 

10(73) 

kick(62) 

king( 83 ) 

kid(71) 

kit( 82 ) 

kin( 63 ) 

kill(75) 

11(67) 

late(79) 

lake( 65 ) 

lay(80) 

lace(96) 

lane(59) 

lame(21) 

12(67) 

map( 50 ) 

mat(76) 

math(57) 

man(67) 

mass( 97 ) 

mad(54) 

13(73) 

page(69) 

pane(73) 

pace(96) 

pay(78) 

pale(61) 

pave( 63 ) 

14(71) 

pas3(97) 

pat( 83 ) 

pack(52) 

pad(69) 

path(54) 

pan(69) 

15(79) 

peace(87) 

peas(88) 

peak( 63 ) 

peal(70) 

peat(81) 

peach(86) 

16(67) 

Pill(71) 

pick(64) 

pip(46) 

Pig (78) 

pin(66) 

plt(80) 

17(68) 

pun(65) 

puff( 70 ) 

pup(53) 

puck(54) 

pus( 98 ) 

pub( 69 ) 

18(79) 

rave(60) 

rake( 63 ) 

race(96) 

rate( 87 ) 

raze( 89 ) 

ray(79) 

19(61) 

sake(72) 

sale(64) 

save(64) 

sane(58) 

safe(77) 

same(32) 

20(70) 

sad(64) 

sass(99) 

sag(63) 

sack(65) 

sap(55) 

sat(73) 

21(49) 

seep(48) 

seen( 52 ) 

seethe(44)seed(45) 

seem( 26 ) 

seek( 80 ) 

22(67) 

sing(76) 

sit(72) 

sin( 65 ) 

sip(46) 

sick( 72 ) 

sill(70) 

23(64) 

sud(72) 

sum(43) 

sub(71) 

sun(72) 

sup(72) 

sung(54) 

24(64) 

tab(57) 

tan(54) 

tam(6l) 

tang(8l) 

tack(63) 

tap(70) 

25 ( 75) 

teach(8l) 

tear(74) 

teaBe(92) 

teal(65) 

team(70) 

teak(68) 
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Orders 
123 
43 17 36 
35 34 50 
27 48 17 
41 21 40 
37 30 44 
4 8 22 
46 11 30 
39 25 43 
14 29 47 
18 37 13 
2 4 18 
32 40 9 
10 20 35 
50 3 21 
24 47 12 
8 16 31 
16 33 3 
20 41 6 
31 42 11 
22 49 14 
45 13 32 
29 44 15 
6 12 27 
12 24 39 
48 7 25 



Section 2: Variation of Initial Element 


Forms __ Orders 



A(78) 

B( 8 l) 

C(79) 

0(79) 

E(78) 

P( 8 l) 

12 3 

1 ( 78 ) 

led(67) 

shed( 98 ) 

red( 66 ) 

bed(74) 

fed( 82 ) 

wed( 78 ) 

47 9 28 

2(85) 

sold(97) 

told(99) 

hold(60) 

fold( 89 ) 

gold(78) 

cold( 85 ) 

1 2 16 

3(72) 

dig(67) 

wig(77) 

big(66) 

rig(54) 

Plg(79) 

fig(90) 

42 19 38 

4(86) 

kick (7 9) 

lick(73) 

sick(98) 

pick(88) 

wick( 82 ) 

tick(94) 

28 46 1 

5(79) 

book(62) 

took(97) 

shook(95) 

cook( 82 ) 

hook( 65 ) 

look(75) 

19 39 4 

6(67) 

hark( 63 ) 

dark(73) 

mark(60) 

lark(54) 

park( 78 ) 

bark(73) 

38 28 46 

7(82) 

gale(79) 

male(66) 

tale(86) 

bale(74) 

sale(100) 

pale(84) 

23 45 10 

8(77) 

peel(81) 

reel(76) 

feel( 85 ) 

heel(74) 

keel(81) 

eel(63) 

13 27 45 

9(78) 

will(76) 

hill(61) 

kill(86) 

till(93) 

fill(88) 

bill(66) 

21 43 8 

10(83) 

foil(80) 

coil(84) 

boil(76) 

oil(76) 

toil( 83 ) 

soil( 98 ) 

40 23 42 

11(86) 

fame(80) 

same(99) 

came( 85 ) 

name(74) 

tame( 90 ) 

game(86) 

30 37 41 

12(73) 

ten(93) 

pen(86) 

den( 65 ) 

hen(6l) 

then(66) 

men(68) 

36 32 48 

13(88) 

pin(85) 

sin( 98 ) 

tin( 92 ) 

win(84) 

din(79) 

fin(90) 

5 10 24 

14(80) 

sun(99) 

nun(70) 

gun(75) 

fun(87) 

bun( 65 ) 

run( 82 ) 

26 50 19 

15 ( 82 ) 

rang(79) 

fang( 89 ) 

gang(74) 

bang(76) 

sang(99) 

hang( 78 ) 

33 38 7 

16(84) 

tent(92) 

bent(8l) 

went(80) 

dent(76) 

rent(73) 

3ent(99) 

25 1 26 

17(83) 

sip(98) 

rip(75) 

tip( 92 ) 

dip( 81 ) 

hlp(79) 

lip(72) 

34 36 5 

18(81) 

top(76) 

hop(79) 

pop(79) 

cop(3o) 

mop(73) 

shop( 96 ) 

11 22 37 

19(72) 

meat( 58 ) 

feat(89) 

heat( 69 ) 

seat(99) 

beat(68) 

neat( 51 ) 

17 35 2 

20 ( 82 ) 

kit(84) 

bit(74) 

fit(84) 

sit(98) 

wit(75) 

hit(77) 

49 5 23 

21(77) 

hot(75) 

got(78) 

not( 72 ) 

pot(75) 

lot(69) 

tot(93) 

7 14 29 

22(72) 

nest(64) 

vest(6l) 

west(75) 

test(94) 

best(71) 

rest(66) 

44 15 34 

23(78) 

bust( 78 ) 

just( 90 ) 

rust(76) 

must(71) 

gust(73) 

dust(80) 

3 6 20 

24(80) 

raw(63) 

paw(82) 

law(70) 

jaw( 85 ) 

thaw(8l) 

saw(97) 

15 31 49 

25(83) 

way(8l) 

may(72) 

say( 98 ) 

gay( 83 ) 

day(75) 

pay(89) 

9 18 33 
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Appendix II 


The materials presented to the listeners included standard in¬ 
structions and various response forms. The instructions were given 
in the written form shown here and were also presented orally with 
the listeners following along on their own copies. 

The six response forms are labeled IX, 1Y, 2X, 2Y, 3X, and 3Y. 

Forms with the same number have the same order of ensembles. Each 
order (i.e., form number) has two word arrangements within ensembles, 
an X and Y designation. In addition, the arrangement of words within 
ensembles is never the same for any two X or Y forms, that is, the 
spatial arrangement of words within ensembles is different for each 
of the six forms. 
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INS T R U CTION S 


You are going to hear some one syllable words presented with different 
loudness levels of noise. Each word will be presented in a carrier 
phrase giving its particular item number. For example: 

Number one Is tree . 

Number two is mile . 

The word presented will be one of the six words which are printed 
on your answer sheet for that particular item. Your task is to 
identify the word presented by drawing a line through the word you 
hear. For example: 

Number three is tow. _ 

row low 

mow sow bow 


Some words will be easier to hear than others. If you are not sure 
what the word is -- guess. Always draw a line through one of the 
six words for each item number. 

Are there any questions? 


The above instructions were presented orally, the subjects following 
along on their own copy of the instructions. 
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NAME 


DATE 


FORM_ IX 


.TEST NO, 
SCORE 


lick 

pick 

tick 

14 

sad 

sass 

sag 

wick 

sick 

kick 

sat 

sap 

sack 


seat meat 

beat 

15 

sip 

sing 

sick I 

heat neat 

feat 

sin 

sill 

sit | 


pus 

pup 

pun 

16 

sold 

told 

hold 

puff 

puck 

pub 

cold 

gold 

fold 


27 


28 


29 


sung 

sup 

sun 

40 

cave 

cane 

came 

sud 

sum 

sub 

cape 

cake 

case 


red 

wed 

shed 

41 

game 

tame 

name 

bed 

led 

fed 

fame 

same 

came 


hot 

got 

not 

42 

oil 

foil 

toil 

tot 

lot 

pot 

boil 

soil 

coll 


look 

hook 

cook 


buck 

but 

bun 

30 

dud 

dub 

dun 

43 

fin 

fit 

fig 

book 

took 

shook 

17 

bU3 

buff 

bug 

dug 

dung 

duck 

fizz 

fill 

fib 


tip 

lip 

rip 

18 

lake 

lace 

lame 

31 

pip 

pit 

pick 

dip 

sip 

hip 

lane 

lay 

late 

pig 

pill 

pin 


rate 

rave 

raze 

19 

gun 

run 

nun 

race 

ray 

rake 

fun 

sun 

bun 


32 


seem seethe seep 
seen seed seek 


bang 

rang 

sang 

20 

rust 

dust 

just 

33 

day 

say 

way 

gang 

hang 

fang 

must 

bust 

gust 

may 

gay 

pay 


44 


45 


46 


cut cub cuff 

cuss cud cup 


feel 

ee 1 

reel 

hee 1 

peel 

kee 1 


dark 

lark 

bark 

park 

mark 

hark 


hill 

till 

bill 

2 1 

pan 

path 

pad 

fill 

kill 

will 

pass 

pat 

pack 


10 


mat 

man 

mad 

mass 

math 

map 


tale 

bale 

pale 

gale 

male 

sale 


sake 

sale 

save 

same 

safe 

sane 


peat 

peas 

peak 

peal 

peace 

peach 


king 

kin 

kit 

kid 

kill 

kick 


22 


23 


24 


25 


26 


dim 

dig 

dill 

did 

din 

dip 


34 


35 


rest best test 

47 

heap 

heat 

heave 

nest vest west 

hear 

heath 

heal 


pane 

pay 

pave 

48 

men 

then 

hen 

pale 

pace 

page 

ten 

pen 

den 


wit 

fit 

kit 

36 

bat 

bad 

back 

49 

raw 

paw 

law 

bit 

sit 

hit 

bath 

ban 

bass 

saw 

thaw 

jaw 


din 

tin 

pin 

37 

cop 

top 

mop 

50 

bead 

beat 

bean 

sin 

win 

fin 

pop 

shop 

hop 

beach 

beam 

beak 


Leal teach team 

38 

fig 

Pig 

rig 

tease teak tear 

__ 

dig 

wig 

big 


tent 

bent 

went 

39 

tap 

tack 

tang 

sent 

rent 

dent 

tab 

tan 

tam 
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NAME 


DATE 


FORM 1Y 


.TEST NO., 
SCORE 


kick lick sick 
tick wick pick 


sack sad sap 

sag sat sass 


neat beat seat 
meat feat heat 


15 


sit sip sill 
sick sin sing 


2 ? 


28 


sup sub sud 

sum sun sung 


40 


cake 

came 

cave 

cane 

case 

cape 


wed fed bed 
led shed red 


tame 

came 

fame 

same 

name 

game 


3 

pun 

puff 

pup 

16 

fold 

sold 

gold 

29 

pot 

hot 

lot 

42 

toil 

boil 

foil 


pub 

pus 

puck 


hold 

cold 

told 


not 

tot 

got 


coil 

oil 

soil 


hook 

shook 

book 

17 

but 

bug 

bus 

30 

duck 

dud 

dung 

43 

fig 

fizz 

fit 

took 

cook 

look 

buff 

bun 

buck 

dun 

dug 

dub 

fib 

fin 

fill 


5 

lip 

hip 

dip 

18 

late 

lake 

lay 

3J 

pit 

pin 

Pig 

44 

cuss cud 

cup 


sip 

rip 

tip 

lame 

lane 

lace 


pill 

pick 

Pip 

cut cub 

cuff 


rake 

rate 

ray 

19 

run 

bun 

fun 

32 

seethe 

seek seen 

45 

heel 

peel 

keel 

raze 

race 

rave 

sun 

nun 

gun 

seed 

seep seem 

feel 

eel 

reel 


fang 

bang 

hang 

20 

dust 

gust must 

33 

say 

pay 

may 

46 

mark 

bark 

dark 

sang 

gang 

rang 

bust 

just rust 

gay 

way 

day 

lark 

hark 

park 


will hill kill 
bill fill till 


map mat math 

mad mass man 


10 


I I 


12 


13 


pale sale bale 
gale male tale 


sane sake safe 
save same sale 


peak peach peas 
peal peace peat 


kin kid kick 
king kit kill 


2 I 


22 


23 


24 


25 


26 


path pack pass 
pat pad pan 


dip dim 
dill did 


din 

dig 


fit 

sit 


hit 

kit 


bit 

wit 


tin 

win 


fin 

pin 


s in 
din 


tear teal teak 
team tease teach 


dent tent rent 
went sent bent 


34 


35 


36 


37 


38 


39 


best rest nest 
vest test rest 


page pane pace 
pave pale pay 


bass bat ban 
back bath bad 


hop 

mop 


cop 

pop 


shop 

top 


47 


48 


49 


50 


heath heave heap 
heat heal hear 


then 

den 

fen 

pen 

hen 

men 


law 

saw 

paw 

jaw 

raw 

thaw 


beat 

beak 

beach 

beam 

bean 

bead 


dig wig big 
fig pig rig 


tack tam tab 
tan tang tap 
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seed seep seem 
seethe seek seen 


fame same came 
game tame name 























































































































































NAME 


DATE 


TEST NO. 


2 lace 
lay 


3 gust 


late lane 
lame lake 


ruat bust 
must dust 


I 7 


tam tap tan 

tang tab tack 


FORM_ 3Y 


cold 

gold 

fold 

sold 

told 

hold 


heath heave heap 


jaw raw thaw 

law saw paw 


bug 

buck 

buff 

bun 

bus 

but 


SCORE 

1 J 


foil 

coil 

boil 

soil 

toil 

oil 


pick kick wick 
sick tick lick 


case cave cake 
came cape cane 


pup 

pub 

puff 

puck 

pun 

pus 


sing 

sit 

8 in 

sill 

sick 

sip 


big 

fig 

wig 

rig 

dig 

pig 


dig 

dip 

did 

din 

dill 

dim 


feat 

seat 

neat 

beat 

heat 

meat 


came 

game 

same 

name 

fame 

tame 


bad 

bass 

bath 

ban 

back 

bat 


fin 

din 

win 

pin 

sin 

tin 


kick 

king 

kid 

kill 

kin 

kit 


sake 

sale 

save 

same 

safe 

sane 


west 

rest 

vest 

test 

nest 

best 


sub 

sung 

sum 

sun 

sud 

sup 


book 

took 

shook 

look 

hook 

cook 


math 

mad 

mat 

man 

map 

mass 


seek Beem seed 
seep seen seeth* 


got 

pot 

tot 

lot 

not 

hot 


raze 

race 

rave 

rake 

rate 

ray 


sang 

gang 

rang 

fang 

bang 

hang 


dub 

duck 

dug 

dung 

dun 

dud 


pin 

Pip 

pill 

pick 

Pig 

pit 


. kill 

bill 

hill 

till 

will 

fill 


rip 

dip 

lip 

hip 

tip 

sip 


fed 

red 

led 

shed 

bed 

wed 


pay 

day 

gay 

way 

may 

say 


sat 

sap 

sack 

sad 

sass 

sag 


beam bean bead 
beat beak beach 


teach 

tear 

tease 

teak 

team 

teal 


pay 

page 

pale 

pace 

pave 

pane 


male 

bale 

pale 

sale 

tale 

gale 


ten 

pen 

den 

men 

then 

hen 


hit 

wit 

sit 

kit 

bit 

fit 


top 

hop 

pop 

shop 

mop 

cop 


peak peach peas 

bent dent sent 
rent went tent 


cub 

cup 

cuss 

cud 

cuff 

cut 


pack 

pan 

pat 

pad 

pass 

path 


lark 

hark 

park 

mark 

bark 

dark 


ee 1 

kee 1 

hee 1 

peel 

ree 1 

fee 1 


bun 

gun 

sun 

nun 

fun 

run 


fib 

fin 

fill 

fig 

fizz 

fit 


V 

m 
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Appendix III 


The word ensembles are arranged in matrix form and responses are 
tabulated for each stimulus, providing a display of confusions as 
well as correct responses. Ihe stimulus words identify the rows at 
the left side of the matrix and the response words (provided by the 
subject's response forms) are the column headings in the matrix. 

The entries in each matrix represent number of responses. Row totals 
are not equal but vary systematically between 432 and 437; matrix 
totals are 2604 in all cases. 

The data are pooled over talkers, listeners, and levels; (each word, 
of course, is drawn from a different form of the test, but the test 
forms are not derivable from these matrices in terms of row place¬ 
ment). The same number of tests were presented at each level and the 
contribution of talkers to the various levels is approximately equal. 
Each form is represented equally in the data. 

The ensemble numbers refer to the materials of Appendix I. 
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NO. PI 

bat 

bad 

back 

bath 

ban 

bass 

bat 

305 

15 

49 

40 

17 

11 

bad 

38 

295 

18 

45 

39 

0 

back 

112 

21 

236 

31 

20 

13 

bath 

94 

31 

31 

259 

8 

11 

ban 

23 

52 

12 

28 

302 

16 

bass 

1 

1 

2 

4 

1 

423 

No. P2 

bead 

beat 

bean 

beach 

beam 

beak 

bead 

236 

45 

95 

11 

15 

31 

beat 

14 

329 

24 

24 

8 

34 

bean 

108 

43 

206 

12 

47 

21 

beach 

10 

32 

14 

360 

7 

12 

beam 

80 

19 

245 

14 

59 

15 

beak 

15 

120 

26 

6 

11 

256 

No. F3 

buck 

but 

bun 

bus 

buff 

bug 

buck 

243 

58 

17 

2 

94 

19 

but 

71 

273 

21 

6 

36 

26 

bun 

24 

38 

316 

12 

22 

25 

bus 

2 

0 

1 

431 

1 

0 

buff 

48 

41 

16 

6 

307 

14 

bug 

22 

25 

34 

4 

22 

327 

No. F4 

cave 

cane 

came 

cape 

cake 

case 

cave 

298 

27 

28 

31 

39 

10 

cane 

32 

216 

102 

33 

39 

11 

came 

34 

170 

167 

30 

26 

10 

cape 

21 

13 

15 

238 

145 

3 

cake 

16 

6 

4 

86 

312 

8 

case 

2 

2 

1 

4 

3 

422 
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No. F5 

cut 

cub 

cuff 

cuss 

cud 

1 

cup 1 

cut 

350 

9 

25 

10 

19 

24 

cub 

4o 

276 

44 

6 

36 

33 | 

cuff 

31 

9 

301 

11 

6 

74 

cuss 

3 

0 

2 

422 

4 

2 I 

cud 

46 

34 

31 

4 

303 

1 

15 

cup 

47 

30 

50 

11 

11 

283 I 

No. F6 

dim 

dig 

dill 

did 

din 

dip 1 

dim 

279 

21 

17 

21 

80 

14 1 

dig 

22 

310 

17 

33 

43 

12 

dill 

25 

27 

291 

47 

25 

18 | 

did 

21 

61 

24 

268 

51 

8 

din 

39 

20 

18 

24 

320 

13 

dip 

21 

18 

10 

19 

20 

347 

No. 57 

dud 

dub 

dun 

dug 

dung 

duck 

dud 

295 

9 

23 

70 

19 

19 

dub 

39 

111 

29 

197 

36 

21 

dun 

24 

9 

297 

24 

61 

19 

dug 

46 

24 

31 

275 

37 

19 

dung 

16 

14 

77 

34 

272 

20 

duck 

16 

19 

17 

29 

25 

331 

No. F8 

fin 

fit 

fig 

fizz 

fill 

fib 

fin 

307 

4o 

35 

18 

14 

19 

fit 

15 

355 

20 

13 

11 

20 

fig 

26 

23 

350 

10 

11 

15 

fizz 

6 

5 

12 

4o4 

1 

4 

fill 

22 

37 

18 

9 

319 

32 

fib 

56 

68 

31 

18 

26 

234 
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No. pq 

heap 

heat 

heap 

132 

200 

heat 

16 

374 

heave 

12 

36 

hear 

19 

40 

heath 

36 

120 

heal 

27 

50 

No. P10 

king 

kit 

king 

359 

9 

kit 

8 

355 

kill 

16 

32 

kin 

63 

32 

kid 

35 

26 

kick 

38 

82 

No. Pll 

lake 

lace 

lake 

284 

3 

lace 

7 

414 

lame 

15 

6 

lane 

16 

8 

lay 

25 

1 

late 

30 

17 

No. F12 

mat 

man 

mat 

332 

25 

man 

36 

289 

mad. 

40 

103 

masB 

5 

2 

math 

90 

23 

map 

128 

33 


heave 

hear 

heath 

heal 

14 

11 

65 

12 

4 

8 

21 

10 

306 

13 

43 

24 

9 

319 

20 

30 

18 

14 

236 

11 

28 

32 

27 

269 

kill 

kin 

kid 

kick 

9 

29 

25 

4 

17 

12 

13 

27 

324 

20 

29 

13 

8 

273 

35 

22 

18 

21 

308 

25 

15 

14 

17 

271 

lame 

lane 

lay 

late 

4 

3 

18 

123 

0 

1 

4 

6 

91 

242 

4l 

39 

69 

256 

52 

32 

6 

26 

346 

29 

7 

16 

21 

346 

mad 

mass 

math 

map 

15 

7 

33 

23 

59 

16 

21 

11 

235 

4 

38 

13 

1 

422 

1 

2 

22 

35 

245 

18 

20 

14 

25 

217 
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No. F13 

pane 

pay 

pave 

pale 

pace 

page 

pane 

316 

51 

23 

16 

8 

21 

pay 

33 

336 

24 

21 

4 

14 

pave 

17 

101 

272 

18 

9 

17 

pale 

12 

99 

30 

263 

12 

17 

pace 

2 

5 

3 

4 

415 

4 

page 

22 

51 

40 

12 

11 

301 

No. Fl4 

pan 

path 

pad 

pass 

pat 

pack 

pan 

299 

27 

44 

4 

44 

16 

path 

8 

235 

13 

60 

96 

21 

pad 

25 

51 

298 

11 

33 

14 

pass 

6 

1 

4 

422 

2 

2 

pat 

3 

20 

13 

2 

360 

37 

pack 

15 

34 

21 

11 

128 

224 

No. F 15 

peat 

peak 

peace 

peas 

peal 

peach 

peat 

351 

42 

5 

9 

9 

17 

peak 

103 

272 

9 

15 

24 

10 

peace 

35 

5 

380 

8 

3 

6 

peas 

16 

5 

15 

381 

5 

13 

peal 

59 

45 

9 

9 

304 

6 

peach 

22 

16 

7 

8 

7 

374 

No. F16 

pip 

pit 

pick 

Pig 

pill 

pin 

Pip 

200 

105 

48 

31 

32 

17 

pit 

10 

348 

32 

22 

11 

11 

pick 

23 

73 

277 

36 

12 

14 

Pig 

25 

22 

18 

336 

7 

24 

pill 

17 

54 

23 

23 

308 

12 

pin 

7 

52 

36 

41 

10 

287 
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No. P17 

pus 

pup 

pun 

puff 

puck 

pub 

pus 

423 

0 

1 

1 

2 

6 

pup 

6 

231 

11 

97 

56 

32 

pun 

6 

29 

285 

33 

44 

40 

puff 

6 

47 

12 

303 

44 

22 

puck 

7 

51 

17 

100 

234 

23 

put 

3 

23 

26 

60 

21 

301 

No. Fl 8 

rate 

rave 

raze 

race 

ray 

rake 

rate 

377 

11 

6 

6 

15 

17 

rave 

40 

264 

24 

8 

82 

19 

raze 

16 

3 

387 

13 

6 

8 

race 

3 

1 

7 

415 

3 

4 

ray 

37 

26 

10 

6 

343 

12 

rake 

107 

15 

11 

3 

26 

273 

No. Fig 

sake 

sale 

save 

same 

safe 

sane 

sake 

314 

10 

21 

5 

73 

14 

sale 

25 

279 

50 

13 

51 

17 

save 

34 

11 

275 

29 

65 

19 

same 

11 

14 

25 

139 

27 

218 

safe 

68 

4 

12 

9 

334 

6 

sane 

22 

8 

33 

97 

23 

249 

No. F20 

sad 

sass 

sag 

sat 

sap 

sack 

sad 

28 l 

4 

53 

50 

20 

29 

sass 

2 

430 

0 

2 

0 

1 

sag 

86 

6 

272 

29 

9 

32 

sat 

13 

13 

6 

317 

25 

60 

sap 

23 

8 

14 

92 

237 

59 

sack 

13 

6 

19 

75 

37 

282 
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No. P21 

3eem 

seethe 

seep 

seen 

seed 

seek 

seem 

114 

38 

14 

175 

40 

52 

seethe 

21 

189 

23 

52 

92 

56 

seep 

7 

34 

210 

13 

20 

153 

seen 

52 

38 

7 

225 

64 

49 

seed 

31 

91 

11 

80 

194 

25 

seek 

3 

35 

26 

13 

12 

345 

No. F22 

sip 

sing 

sick 

sin 

sill 

sit 

sip 

199 

15 

40 

15 

13 

151 

sing 

10 

333 

4l 

25 

2 

26 

sick 

21 

23 

310 

10 

4 

64 

sin 

25 

47 

37 

280 

11 

33 

sill 

16 

20 

42 

17 

304 

35 

sit 

16 

9 

79 

11 

6 

314 

No. P23 

sung 

sup 

sun 

sud 

sum 

BUb 

sung 

232 

15 

106 

17 

49 

15 

sup 

27 

313 

15 

15 

18 

45 

sun 

41 

20 

312 

17 

23 

19 

sud 

12 

31 

27 

316 

11 

40 

sum 

131 

19 

60 

11 

187 

27 

sub 

26 

30 

16 

38 

16 

307 

No. F24 

tap 

tack 

tang 

tab 

tan 

tam 

tap 

305 

74 

12 

15 

16 

12 

tack 

79 

272 

27 

15 

14 

26 

tang 

5 

11 

350 

9 

34 

23 

tab 

34 

36 

37 

250 

32 

48 

tan 

12 

9 

70 

13 

234 

97 

tam 

16 

18 

38 

19 

80 

262 
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No. F25 

teal 

teach 

teal 

281 

9 

teach 

5 

35^ 

team 

33 

15 

tease 

3 

12 

teak 

14 

10 

tear 

44 

3 


team 

tease 

teak 

tear 

46 

7 

63 

26 

17 

7 

43 

11 

303 

24 

50 

8 

6 

399 

ll 

2 

25 

80 

293 

12 

24 

6 

48 

320 
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No. 11 

red 

wed 

red 

286 

65 

wed 

47 

339 

shed 

1 

1 

bed 

33 

44 

led 

41 

56 

fed 

28 

25 

No. 12 

sold 

told 

sold 

425 

3 

told 

0 

429 

hold 

3 

14 

cold 

3 

11 

gold 

11 

12 

fold 

7 

12 


No. 13 

fig 

Pig 

fig 

391 

9 

Pig 

21 

342 

rig 

15 

20 

dig 

24 

28 

wig 

8 

13 

big 

13 

4o 

No. 14 

lick 

Pick 

lick 

319 

12 

pick 

12 

382 

tick 

2 

7 

wick 

15 

40 

sick 

2 

2 

kick 

15 

23 


shed 

bed 

led 

fed 

11 

26 

21 

24 

6 

12 

19 

11 

428 

0 

3 

2 

8 

320 

14 

13 

9 

26 

294 

11 

5 

10 

9 

356 

hold 

cold 

gold 

fold 

0 

3 

3 

3 

3 

1 

1 

1 

258 

101 

45 

12 

20 

369 

16 

15 

19 

30 

339 

22 

9 

15 

3 

386 

rig 

dig 

wig 

big 

3 

4 

13 

14 

10 

8 

22 

30 

233 

4 

133 

27 

6 

291 

37 

51 

27 

4 

333 

50 

24 

15 

57 

284 

tick 

wick 

sick 

kick 

16 

61 

10 

17 

3 

19 

7 

9 

406 

4 

7 

8 

10 

355 

4 

9 

1 

0 

426 

2 

17 

20 

17 

345 
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No. 15 

look 

hook 

cook 

book 

took 

shook 

\ 

j 

look 

326 

30 

13 

38 

17 

10 


hook 

44 

283 

42 

28 

27 

9 

i 

i 

cook 

21 

19 

352 

15 

19 

6 

1 

book 

89 

30 

19 

270 

14 

15 

i 

took 

5 

2 

6 

0 

420 

2 


shook 

5 

6 

4 

5 

4 

409 


No. 16 

dark 

lark 

bark 

park 

mark 

hark 


dark 

317 

44 

34 

9 

17 

14 


lark 

18 

231 

72 

26 

55 

30 


bark 

10 

29 

318 

26 

41 

10 

i 

* 

park 

5 

11 

32 

338 

23 

24 


mark 

20 

27 

64 

40 

259 

23 


hark 

3 

15 

23 

89 

32 

275 


No. 17 

tale 

pale 

male 

bale 

gale 

sale 

1 

tale 

371 

20 

7 

10 

16 

9 


pale 

11 

365 

13 

12 

25 

8 

■i 

male 

11 

23 

289 

73 

27 

12 


bale 

6 

23 

49 

321 

24 

9 

\ 

gale 

14 

17 

20 

28 

345 

13 


sale 

0 

0 

0 

2 

0 

431 


No. 18 

feel 

eel 

reel 

heel 

peel 

keel 

j 

feel 

366 

5 

14 

16 

14 

18 

i 

eel 

24 

273 

22 

73 

22 

20 


reel 

23 

31 

330 

20 

19 

12 

i 

heel 

18 

48 

10 

321 

21 

14 


peel 

19 

20 

16 

14 

354 

14 

f 

keel 

19 

10 

5 

33 

15 

351 


I 
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No. 19 

hill 

till 

bill 

hill 

265 

18 

30 

till 

3 

401 

12 

bill 

16 

11 

288 

fill 

8 

10 

6 

kill 

18 

11 

10 

will 

24 

20 

22 

no. no 

oil 

foil 

toil 

oil 

328 

24 

14 

foil 

12 

350 

18 

toil 

10 

10 

360 

boil 

41 

25 

14 

soil 

1 

3 

1 

coil 

19 

25 

14 


No. Ill 

game 

tame 

name 

game 

372 

3 

26 

tame 

14 

390 

4 

name 

33 

12 

318 

fame 

12 

9 

18 

same 

1 

1 

1 

came 

23 

12 

14 

No. 112 

men 

then 

hen 

men 

297 

56 

19 

then 

74 

284 

13 

hen 

41 

18 

262 

ten 

6 

7 

5 

pen 

26 

11 

10 

den 

34 

78 

9 


fill 

kill 

will 

50 

43 

29 

2 

11 

3 

19 

22 

78 

379 

17 

13 

15 

371 

8 

17 

20 

334 

boil 

soil 

coil 

33 

6 

27 

14 

5 

38 

10 

4 

39 

329 

6 

18 

1 

426 

2 

7 

3 

367 

fame 

same 

came 

2 

11 

20 

6 

10 

9 

28 

10 

31 

350 

9 

39 

1 

431 

0 

10 

6 

368 

ten 

pen 

den 

12 

39 

11 

14 

27 

21 

21 

77 

13 

406 

6 

7 

4 

376 

8 

17 

15 

280 
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No. 113 

din 

tin 

din 

344 

16 

tin 

9 

396 

pin 

10 

12 

sin 

0 

2 

win 

14 

7 

fin 

7 

12 

No. 114 

gun 

run 

gun 

322 

32 

run 

17 

356 

nun 

60 

36 

fun 

27 

11 

sun 

0 

2 

bun 

31 

75 

No. 115 

bang 

rang 

bang 

326 

41 

rang 

28 

343 

sang 

1 

1 

gang 

25 

39 

hang 

22 

9 

fang 

14 

18 

No. 116 

tent 

bent 

tent 

404 

6 

bent 

6 

351 

went 

15 

18 

sent 

1 

0 

rent 

7 

20 

dent 

10 

35 


pin 

sin 

win 

fin 

19 

8 

37 

9 

3 

11 

8 

6 

373 

15 

8 

19 

1 

425 

1 

6 

20 

13 

363 

15 

7 

5 

13 

390 

nun 

fun 

sun 

bun 

28 

16 

11 

24 

27 

14 

11 

10 

303 

11 

3 

21 

6 

380 

4 

4 

2 

0 

432 

1 

17 

18 

11 

281 

sang 

gang 

hang 

fang 

3 

21 

19 

22 

10 

18 

20 

18 

427 

1 

1 

2 

6 

319 

26 

18 

7 

19 

338 

39 

3 

3 

8 

389 

went 

sent 

rent 

dent 

7 

5 

7 

8 

32 

3 

31 

12 

348 

12 

34 

6 

1 

431 

0 

1 

63 

16 

316 

11 

26 

8 

24 

329 
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No. 117 

tip 

lip 

rip 

dip 

sip 

hip 

tip 

399 

1 

8 

14 

6 

5 

lip 

12 

314 

38 

22 

9 

39 

rip 

9 

44 

324 

23 

13 

22 

dip 

6 

24 

23 

350 

7 

22 

sip 

0 

3 

3 

2 

427 

2 

hip 

24 

28 

19 

15 

6 

341 

No. 118 

cop 

top 

mop 

pop 

shop 

hop 

cop 

344 

12 

7 

12 

4 

53 

top 

46 

330 

13 

12 

5 

31 

mop 

24 

7 

318 

21 

7 

56 

pop 

21 

9 

26 

343 

6 

28 

shop 

7 

0 

2 

0 

418 

7 

hop 

34 

12 

17 

28 

1 

343 

No. 119 

seat 

meat 

beat 

heat 

neat 

feat 

seat 

427 

0 

1 

2 

1 

1 

meat 

6 

252 

81 

37 

41 

20 

heat 

4 

62 

295 

27 

22 

23 

heat 

8 

31 

27 

298 

28 

41 

neat 

5 

93 

48 

47 

221 

20 

feat 

4 

9 

12 

24 

1 

385 

No. 120 

wit 

fit 

kit 

bit 

Sit 

hit 

wit 

326 

33 

15 

30 

11 

18 

fit 

16 

364 

22 

13 

4 

14 

kit 

7 

26 

365 

8 

4 

27 

bit 

60 

13 

10 

322 

4 

26 

sit 

3 

4 

2 

0 

421 

2 

hit 

16 

40 

18 

21 

4 

335 
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No. 121 

hot 

got 

not 

tot 

lot 

pot 


hot 

329 

25 

27 

12 

11 

33 


got 

15 

338 

56 

11 

12 

3 

\ 

not 

29 

5? 

312 

13 

8 

14 


tot 

9 

11 

6 

403 

1 

4 

j 

lot 

47 

32 

32 

5 

299 

18 

1 

pot 

59 

17 

11 

11 

li 

323 

j 

1 

No. 122 

rest 

best 

test 

nest 

vest 

west 


rest 

287 

25 

12 

18 

26 

66 

1 

test 

?5 

307 

9 

10 

44 

38 


test 

3 

4 

405 

6 

6 

8 

J 

nest 

23 

58 

22 

278 

28 

28 


vest 

29 

63 

11 

30 

267 

35 

j 

west 

47 

15 

15 

14 

18 

324 


No. 123 

rust 

dust 

Just 

must 

bUBt 

gust 

1 

rust 

331 

9 

12 

29 

24 

28 


dust 

15 

346 

8 

15 

16 

34 

i 

just 

4 

12 

390 

9 

6 

14 


must 

25 

15 

9 

308 

53 

22 

1 

bust 

18 

12 

11 

41 

341 

14 


gust 

20 

53 

10 

14 

19 

317 

j 

No. 124 

raw 

paw 

law 

saw 

thaw 

Jaw 

i 

raw 

276 

26 

70 

21 

31 

13 

1 

paw 

17 

356 

27 

1 

27 

7 

a 

law 

53 

43 

301 

6 

19 

11 

i 

saw 

3 

1 

2 

422 

3 

3 


thaw 

ts 

49 

14 

5 

351 

6 


jaw 

12 

9 

21 

4 

20 

366 


I 

i 
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-No- 125 day say 

day 326 6 

say 1 424 

way 21 5 

may 21 3 

gay 24 2 

pay 7 7 


way 

may 

gay 

pay 

13 

22 

47 

19 

2 

0 

3 

3 

352 

15 

15 

29 

40 

314 

19 

38 

13 

15 

357 

21 

13 

12 

7 

388 
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I 
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